Virtualization of graphics resources

ABSTRACT

Graphics resources are virtualized through an interface between graphics hardware and graphics clients. The interface allocates the graphics resources across multiple graphics clients, processes commands for access to the graphics resources from the graphics clients, and resolves conflicts for the graphics resources among the clients.

RELATED APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 10/964,220, filed on Oct. 12, 2004, which is a divisionalapplication of U.S. patent application Ser. No. 10/043,018, filed onJan. 8, 2002, now issued as U.S. Pat. No. 6,809,736. The presentapplication is also related to U.S. patent application Ser. Nos.10/042,882 and 10/042,901, both filed on Jan. 8, 2002 and assigned tothe same assignee as the present application.

FIELD OF THE INVENTION

This invention relates generally to computer graphics, and moreparticularly to virtualizing resources for computer graphics.

COPYRIGHT NOTICE/PERMISSION

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever. The following notice applies to the software and dataas described below and in the drawings hereto: Copyright © 1999-2002,Apple Computer, Inc., All Rights Reserved.

BACKGROUND OF THE INVENTION

A graphics kernel driver typically interfaces between graphics clientdrivers and graphics hardware to assign graphics resources to eachclient driver and to administer the submission of graphics commands tothe graphics hardware. Each client driver has explicit knowledge of thegraphics resources it is assigned and references the resources in itscommands using the physical address of the resources. As moresophisticated graphics features are developed, the demand for graphicsresources is ever increasing but the graphics resources are limited bythe graphics hardware and other system constraints. The assignedresources cannot be shared among clients because the graphics hardwareis not designed to handle resource contention among the clients.Additionally, the client drivers are required to manage their owninternal resource conflicts. For example, they must handle theirattempts to use more than available graphics memory.

SUMMARY OF THE INVENTION

Graphics resources are virtualized through an interface between graphicshardware and graphics clients. The interface allocates the graphicsresources across multiple graphics clients, processes commands foraccess to the graphics resources from the graphics clients, and resolvesconflicts for the graphics resources among the clients.

In one aspect, the interface is a graphics kernel that assigns anidentifier to a resource when allocated by a graphics client and theclient uses the identifier instead of an address for the resource whenrequesting access to the resource.

Because the native command structure for the graphics hardware isunaffected by the virtualization, neither the applications nor thehardware require modification to operate in conjunction with the presentinvention. Furthermore, because the virtualized resources appear asunlimited resources to the graphics clients, the clients can besimplified since, for example, they are no longer required tode-fragment or compact their assigned resources.

The present invention describes systems, methods, and machine-readablemedia of varying scope. In addition to the aspects of the presentinvention described in this summary, further aspects of the inventionwill become apparent by reference to the drawings and by reading thedetailed description that follows.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG 1A is a diagram illustrating a graphics driver stack thatincorporates the present invention;

FIG. 1B is a diagram illustrating a system overview of one embodiment ofprocessing in the driver stack of FIG. 1A;

FIGS. 2A-B illustrate graphics command streams according to oneembodiment of the invention;

FIGS. 3A-C illustrate processing of command buffers according toembodiments of the invention;

FIG. 4A is a flowchart of a graphics client driver method to beperformed by a computer processor according to an embodiment of theinvention;

FIG. 4B is a flowchart of a graphics kernel driver method to beperformed by a graphics processor according to an embodiment of theinvention;

FIG. 5A is a diagram of one embodiment of an operating environmentsuitable for practicing the present invention; and

FIG. 5B is a diagram of one embodiment of a computer system suitable foruse in the operating environment of FIG. 5A.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of embodiments of the invention,reference is made to the accompanying drawings in which like referencesindicate similar elements, and in which is shown by way of illustrationspecific embodiments in which the invention may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention, and it is to be understood thatother embodiments may be utilized and that logical, mechanical,electrical, functional, and other changes may be made without departingfrom the scope of the present invention. The following detaileddescription is, therefore, not to be taken in a limiting sense, and thescope of the present invention is defined only by the appended claims.

In one embodiment, the present invention is integrated into a graphicsdriver stack 100 as illustrated in FIG. 1A. A graphics kernel driver 101interfaces between graphics client drivers 103, 105, 107, 109 andgraphics hardware 111 to virtualize limited graphics resources used bythe graphics hardware 111 and manage contention among the client driversfor the resources. The virtualized resources appear as unlimitedresources to the client drivers, which allows the client drivers to besimplified since, for example, they are no longer required tode-fragment or compact their assigned memory.

Graphics resources eligible for virtualization include any limitedresource used by the graphics hardware 111, such as graphics memory,either integrated in the graphics hardware 111 or allocated in systemmemory, GART (graphics address re-mapping table) entries, memoryapertures for accessing video memory or registers, specialized memoryareas for hierarchical depth buffers, among others. For the sake ofclarity, the virtualization of graphics memory is used as an examplethroughout, but the invention is not so limited.

Referring now to an exemplary embodiment shown in FIG. 1B, the kerneldriver 101 manages the allocation of memory among clients through avirtualization map 117, such as a range allocation table. Thevirtualization map 117 indicates how graphics memory is currentlyallocated, including which block a client is using.

An application 115 calls an OpenGL engine 113 through an OpenGL API(application program interface) 119 to create an image. The OpenGLengine 113, executing on the central processing unit (CPU) of thecomputer, determines how to divide the image processing work between theCPU and the graphics processor of the graphics hardware 111, and sendsthe commands to be processed by the graphics processor to the OpenGLclient driver 103 through a client driver API 121. The client driver103, also executing on the CPU, evaluates the commands and determinesthat it needs graphics memory to create the image. The client driver 103requests a block of memory from the kernel driver 101 through a kerneldriver API call 123. The kernel driver 101, executing on the graphicsprocessor, records the request in an entry in the virtualization map117, and associates an identifier with the entry. The kernel driver 101returns the identifier to the client driver 103 for use in all commandsthat access the memory block. Because the native command structure forOpenGL and the graphics hardware is unaffected by the virtualization,neither the application 115, the OpenGL engine 113, nor the hardware 111require modification to operate in conjunction with the presentinvention.

In one embodiment, the kernel driver 101 performs the actual physicalallocation of memory when the client driver 103 submits a command thatreferences the identifier. In another embodiment, the kernel driver 101physically allocates the memory upon receiving the allocation request.In either case, when all physical memory is already allocated, thekernel driver 101 pages a corresponding amount of data currently inmemory to a backing store and updates the virtualization map 117. Thekernel driver 101 uses the virtualization map 117 to determine how topage the data back into memory for subsequent processing. Details of thepaging are described further below in conjunction with FIG. 4B.

In one embodiment, the identifier is a “token” that represents thememory block and the client driver 103 creates tokenized commands bysubstituting the token for the memory address. When the client driver103 submits a tokenized command to the graphics hardware 111, the kerneldriver 101 extracts the token, finds the address of the memory blockrepresented by the token in the virtualization map 117, and replaces thetoken with the real address. When the tokenized commands are submittedas part of a standard graphics command stream, the kernel driver 101must parse the stream into its individual commands and evaluate most, ifnot all, the commands to determine which contain tokens. This can be aslow and expensive operation.

Therefore, in another embodiment, the client driver 103 formats thecommand stream as illustrated in FIG. 2B. A command stream 200 containsstandard commands 203, 205, followed by a tokenized command 207,followed by various other commands, and terminates with a tokenizedcommand 209. The stream 200 is prefaced with a “jump” packet 201 thatpoints to the first tokenized command 207 in the stream 200. Thetokenized command 207 contains another jump packet that points to thenext tokenized command in the stream 200, and so on until the last jumppacket in the stream is reached. The jump packets thus create a linkedlist of tokenized commands, allowing the kernel driver 101 to ignore thestandard commands in the stream 200 without having to evaluate eachcommand individually.

In one embodiment, the jump packets contain a packet type and an offsetvalue relative to the current packet. Assuming a command stream 210 asillustrated in FIG. 2B, the kernel driver 101 reads the first command inthe stream, which is a “start” jump packet 211. The kernel driver 101extracts the offset value from the start jump packet 211 and deletes thepacket from the stream. The kernel driver 101 uses the offset value tojump to the next jump packet 219, which is in the “load texture” command217. The kernel driver 101 extracts the next offset value and packettype from the jump packet 219. The packet type identifies the packet 219as a “texture” packet, indicating that the token 221 represents a blockof memory containing texture data. The kernel driver 101 replaces thetokenized command 217 with a valid graphics command 225 containing thememory address 223 corresponding to the token 221, , and jumps to thejump packet in the next tokenized command in the stream. The resultingstream 220 received by the graphics hardware 111 contains “polygon” 213and “change state” 215 commands unchanged from the stream 210 submittedby the client driver 103, and a “load texture” command 225 as modifiedby the kernel driver 101. Thus, the final processing of the commandstream by the kernel driver only requires each jump packet to be readand written to and from memory while the majority of the command datagenerated by the client driver is not read or interpreted by the kerneldriver.

Alternate embodiments in which the jump packets are not embedded in thetokenized commands in the stream or are submitted as a separate streamassociated with the command stream are contemplated as within the scopeof the invention.

When a particular region of graphics memory requested by a currentclient driver has been reused by a previous client driver, the kerneldriver completes the use of the memory by the previous client driver,and prepares the resource for use by the current client driver. When thekernel driver processes a tokenized command, the graphics memoryreferenced by the token may be in one of two states: valid for immediateuse by the client driver or not. If the memory is valid for immediateuse, the kernel driver proceeds as previously described. If the memoryis not valid for immediate use, the kernel driver refreshes the currentclient's data by allocating a new region of graphics memory and page thedata into it. Before doing this however, the kernel driver submits allgraphics commands in the current client's command stream up to thecurrent jump packet to the graphics hardware before it starts allocatingthe new region of graphics memory for the current client because theprocess of allocation might result in the deallocation and paging ofgraphics memory previously referenced in the current command stream.Details of the refreshing of data are described further below inconjunction with FIG. 4B.

Command buffers are commonly used to hold the command streams frommultiple clients. As shown in FIG. 3A, as the client driver generatescommands, the CPU fills the appropriate buffer 301, 303. When a bufferis full, it is placed in a processing queue for the graphics hardware,and the CPU assigns another buffer to the client driver. It will beappreciated that when jump packets are used, the client driver loads thestart jump packet first in the buffer.

The command buffers allow multiple clients to create streamsasynchronously to each other. The command buffers also allow thegraphics hardware and the CPU to operate asynchronously, keeping bothbusy even though they typically operate at different speeds.

In one embodiment, the queued buffers are arranged as a linked list asshown in FIG. 3B. The contents of the buffers 301, 303, 305 are read bythe graphics hardware 111 as a linear stream of commands for executionin a serialized fashion, i.e., all the commands in one buffer areexecuted before the commands in the next buffer in the queue. Theserialized, linear execution by the graphics hardware 111 provides thekernel driver 101 with an memory management timeline to follow inprocessing the commands that reference graphics memory. After processingby the kernel driver, the entire command stream is valid for consumptionby the graphics hardware since the conflicts between clients due toreuse of memory have been resolved and the jump packets and tokenizedcommands have been replaced with valid graphics hardware commands.

In an alternate embodiment, the identifier for the memory blockallocated to the client driver 103 is the physical address of thememory. Because the client expects memory address to be unchanged untilit de-allocates the memory, the kernel driver 101 employs specialgraphics hardware features to manage the virtualization of memory. Inone embodiment, the kernel driver 101 uses graphics semaphores thatcause the graphics hardware to suspend processing of one buffer andswitch to processing another buffer, thus interleaving the processing ofthe command buffers from different clients, and creating multipleinter-dependent linear timelines as illustrated in FIG. 3C.

For example, assume client A places a command in buffer 307 thatreferences memory also used by client C. When the kernel driver 101reaches that command in buffer 307, it inserts a reference to semaphore313 before the command, effectively dividing the buffer 307 into commandsequences 311, 315. The graphics hardware 111 processes command sequence311 in buffer 307 until it reaches semaphore 313, which directs it toswitch to processing the next queued buffer 309. While the graphicshardware 111 is processing buffer 309, the kernel driver 101 pages theappropriate data back in and clears the semaphore 313.

Similarly, assume client B places a command in buffer 309 thatreferences memory also used by client D, so the kernel driver 101inserts a reference to semaphore 321 in buffer 309, creating commandsequences 319, 323. When the graphics hardware 111 reaches semaphore321, it determines that semaphore 313 is clear and resumes processingbuffer 307 at command sequence 315. Because the kernel driver 101 hascleared semaphore 321 by the time the graphics hardware finishesprocessing command sequence 315, the graphics hardware can now processcommand sequence 323.

Next, the particular methods of the invention are described in terms ofcomputer software with reference to a series of flowcharts. The methodsto be performed by a processing system constitute computer programs madeup of executable instructions illustrated as blocks (acts). Describingthe methods by reference to a flowchart enables one skilled in the artto develop such programs including such instructions to carry out themethods on suitably configured hardware (the processing unit of thehardware executing the instructions from machine-readable media). Theexecutable instructions may be written in a computer programminglanguage or may be embodied in firmware logic. If written in aprogramming language conforming to a recognized standard, suchinstructions can be executed on a variety of hardware platforms andinterface to a variety of operating systems. In addition, the presentinvention is not described with reference to any particular programminglanguage. It will be appreciated that a variety of programming languagesmay be used to implement the teachings of the invention as describedherein. Furthermore, it is common in the art to speak of software, inone form or another (e.g., program, procedure, process, application,module, logic . . . ), as taking an action or causing a result. Suchexpressions are merely a shorthand way of saying that execution of thesoftware by a machine causes the processor of the machine to perform anaction or produce a result. It will be further appreciated that more orfewer processes may be incorporated into the methods illustrated inFIGS. 4A-B without departing from the scope of the invention and that noparticular order is implied by the arrangement of blocks shown anddescribed herein.

Referring first to FIG. 4A, the acts to be performed by a computerprocessor executing a client driver method 400 that tokenizes commandsare shown. The client driver method 400 receives an image command (block401) and determines if graphics resources are required to process thecommand (block 403). If the necessary resources have not been previouslyallocated, the method 400 requests the resources from the kernel driver(block 405) and receives a token in return (block 407). The method 400creates the graphics commands to perform the image command at block 409.The processing represented by block 409 includes creating the jumppackets with the appropriate offsets and packet types, and inserting thejump packets and tokens in the commands. The particular packet typesused by embodiments of the invention are dictated by the command set ofthe underlying graphics hardware. One exemplary set of packet types,called “op codes,” for graphics memory are shown in Table 1.

TABLE 1 Op Code Remarks kGLStreamStart Start the stream kGLStreamEndTerminate the stream kGLStreamCopyColor Copy an image between two drawbuffers kGLStreamCopyColorScale Copy an image between two draw bufferswith scaling kGLStreamDrawColor Draw an image to the current draw bufferkGLStreamTexture0 Set the current texture object on texture unit zerokGLStreamTexture1 Set the current texture object on texture unit onekGLStreamTexture2 Set the current texture object on texture unit twokGLStreamTexture3 Set the current texture object on texture unit threekGLStreamNoTex0 Remove any texture from texture unit zerokGLStreamNoTex1 Remove any texture from texture unit one kGLStreamNoTex2Remove any texture from texture unit two kGLStreamNoTex3 Remove anytexture from texture unit three kGLStreamVertexBuffer Set the currentvertex object kGLStreamNoVertexBuffer Remove any current vertex object

If there is no existing command buffer (block 411), the method 400starts a new buffer (block 413) and inserts a start jump packet at thebeginning of the buffer (block 415) with an offset to the firsttokenized command in the buffer. Each graphics command is loaded in thebuffer (block 417) until all graphics commands are buffered (block 419)or the current buffer is full (block 421). If the current buffer is fulland more commands need to be buffered, the method 400 returns to block413 to start a new buffer.

Referring now to FIG. 4B, the acts to be performed by a graphicsprocessor executing a kernel driver method 430 corresponding to theclient driver method 400 are shown. The kernel driver method 430 isillustrated as two parallel processing threads, one that interfaces withthe client driver (starting at block 431) and one that interfaces withthe graphics hardware (starting at block 451). It will be appreciatedthat the invention is not limited to such parallel processingimplementations.

When the method 430 receives an allocation request from a client driver(block 431), it determines if the requested amount of resource isavailable (block 433). If not, the method 430 pages out a sufficientamount of data belonging to another client (block 435). The method 430allocates the resource, including assigning a token and updating itsmemory management information, such as the virtualization map 117illustrated in FIG. 1B. The token is returned to the requesting clientdriver at block 439. The client driver method 430 waits until anotherrequest is received (block 441) and returns to block 431 to process thenew request.

When the client driver submits a buffer of commands to the graphicshardware for processing, the kernel driver method 430 extracts theoffset and type from the next jump packet in the buffer (block 451). Ifthe next jump packet is the first jump packet, i.e., a start jump packet(block 453), the method 430 deletes the start jump packet from thebuffer (block 461) and jumps to the jump packet defined by the offset(block 465) to continue processing. Otherwise, the method 430 uses thejump packet type to locate the token in the command and determines ifthe resource corresponding to the token has been reused (block 455). Ifso, the kernel driver method 430 refreshes the data required by thecurrent command (block 457). Because of the abstraction provided by thetoken, the kernel driver can page the data into a different availablegraphics resource or page out the data currently in the originalresource and page in the data required by the current command. The tokenis replaced with the address of the resource (block 459) and the jumppacket is deleted (block 461). If the current jump packet is the last inthe buffer (block 463), the method 430 waits for another buffer (block467) and returns to block 451 to process the new buffer. Otherwise, thenext jump packet in the buffer is processed.

In an alternate embodiment, the processing represented by block 437 is alogical allocation of the resource to the client driver and theprocessing represented by blocks 433 through 435 is not performed. Thekernel driver method 430 performs the physical allocation, and anynecessary paging, when it encounters the first tokenized command thatreferences the resource in the command buffer.

In one embodiment, the kernel driver method 430 uses system memory asits backing store for data that must be paged out of the virtualizedgraphics resources. The method 430 can request the CPU read the datainto system memory, or it can request the graphics hardware to write thedata to the system memory. The latter operation can be performedasynchronously with the CPU, but not all graphics hardware may be ableto perform the operation or there may be incompatibilities between thegraphics hardware and the CPU. When the operating system virtualizessystem memory, the operating system may further page the data to massstorage. It will be appreciated that once the data has been written tosystem memory, a virtual memory operating system may further page thedata to mass storage.

In one embodiment, what data to page into system memory is determined byvarious paging criteria, such as type of graphics resource, priority,and paging algorithm. Some resources, like graphics memory, are veryexpensive to page because the data contained in the graphics memoryoften must be copied into system memory. The priorities may be allocatedwithin graphics resources types. For example, texture objects generallyhave a lower priority than frame buffers when paging graphics memory.Other resources, like GART entries may be paged inexpensively becausethe paging only requires the modification of the GART table, i.e., nodata is actually relocated. Because the relative cost of pagingdifferent types of resources is quite different, different pagingalgorithms are used for each.

For example, when a client driver requests an allocation of graphicsmemory but there is not enough free contiguous memory to service therequest, all graphics memory resources owned by all clients arecandidates for paging. The first resources selected are owned by otherclients because there may be an arbitrarily long period of time beforethe other clients are run again. When considering graphics memory ownedby the requesting client driver, the kernel driver uses an algorithmthat dynamically switches from LRU (least recently used) to MRU (mostrecently used) based on whether or not the client driver isovercommitted in its texture usage. An overcommitted application is anapplication that uses more texture memory in rendering a single framethan can be supplied by the graphics hardware. When a client driver thatis not overcommitted runs out of graphics memory it is because some userinput has caused the client driver to render a new scene so the LRUalgorithm is used, based on the assumption that the least recently usedmemory resources may never be used again. When a client driver that isovercommitted runs out of graphics memory this means that it will do socyclically every frame, so the MRU algorithm is chosen because an LRUalgorithm would result in every memory resource owned by the clientdriver being paged one or more times per frame.

GART entry paging is managed differently because the cost of changingGART entries is essentially unrelated to the size of the memoryresource. The first candidates for paging are GART entries that maynever be used again. For example, graphics memory texture objects eachhave a GART entry that was used to transfer the texture from systemmemory to graphics memory. Once the texture has been moved to graphicsmemory, the GART entry will never be used again unless the texture ispaged from graphics memory and then reloaded. Therefore, it is likelythat choosing such a GART entry for paging will have no performancecost. The remaining GART entries are categorized from highest to lowestpriority for paging, with the lowest priority assigned to the GART entryfor each client's command buffer, which must be mapped into GART for theclient driver to use the graphics hardware at all.

One of skill in the art will appreciate that other types of graphicsresources may have different algorithms for selecting which resourcesare candidates for paging that allow the resources to be transparentlymanaged with respect to multiple clients as described above for graphicsmemory and GART.

In one embodiment, the kernel driver method 430 uses a collection ofdata objects, each of which represents an allocated resource, as avirtualization map. The tokens identify the data objects within thevirtualization map. Each data object contains the address range for thecorresponding resource. When the data in the resource is paged out, a“dirty” flag is set and a pointer to the backing store holding the datais stored in the object. It will be appreciated that the layer ofabstraction between the client and the physical resources provided bythe token allows the data to be paged into a resource address differentthan it previously occupied without the client driver being aware of thechange.

The following description of FIGS. 5A-B is intended to provide anoverview of computer hardware and other operating components suitablefor performing the methods of the invention described above, but is notintended to limit the applicable environments. One of skill in the artwill immediately appreciate that the invention can be practiced withother processing system configurations, including hand-held devices,multiprocessor systems, microprocessor-based or programmable consumerelectronics, network PCs, minicomputers, mainframe computers, and thelike. The invention can also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network.

FIG. 5A shows several computer systems that are coupled together througha network 3, such as the Internet. The term “Internet” as used hereinrefers to a network of networks which uses certain protocols, such asthe TCP/IP protocol, and possibly other protocols such as, for example,the hypertext transfer protocol (HTTP) for hypertext markup language(HTML) documents that make up the World Wide Web (web). The physicalconnections of the Internet and the protocols and communicationprocedures of the Internet are well known to those of skill in the art.Access to the Internet 3 is typically provided by Internet serviceproviders (ISP), such as the ISPs 5 and 7. Users on client systems, suchas client computer systems 21, 25, 35, and 37 obtain access to theInternet through the Internet service providers, such as ISPs 5 and 7.Access to the Internet allows users of the client computer systems toexchange information, receive and send e-mails, and view documents, suchas documents which have been prepared in the HTML format. Thesedocuments are often provided by web servers, such as web server 9 whichis considered to be “on” the Internet. Often these web servers areprovided by the ISPs, such as ISP 5, although a computer system can beset up and connected to the Internet without that system being also anISP as is well known in the art.

The web server 9 is typically at least one computer system whichoperates as a server computer system and is configured to operate withthe protocols of the World Wide Web and is coupled to the Internet.Optionally, the web server 9 can be part of an ISP which provides accessto the Internet for client systems. The web server 9 is shown coupled tothe server computer system 11 which itself is coupled to web content 10,which can be considered a form of a media database. It will beappreciated that while two computer systems 9 and 11 are shown in FIG.5A, the web server system 9 and the server computer system 11 can be onecomputer system having different software components providing the webserver functionality and the server functionality provided by the servercomputer system 11 which will be described further below.

Client computer systems 21, 25, 35, and 37 can each, with theappropriate web browsing software, view HTML pages provided by the webserver 9. The ISP 5 provides Internet connectivity to the clientcomputer system 21 through the modem interface 23 which can beconsidered part of the client computer system 21. The client computersystem can be a personal computer system, a network computer, a Web TVsystem, or other such computer system. Similarly, the ISP 7 providesInternet connectivity for client systems 25, 35, and 37, although asshown in FIG. 5A, the connections are not the same for these threecomputer systems. Client computer system 25 is coupled through a modeminterface 27 while client computer systems 35 and 37 are part of a LAN.While FIG. 5A shows the interfaces 23 and 27 as generically as a“modem,” it will be appreciated that each of these interfaces can be ananalog modem, ISDN modem, cable modem, satellite transmission interface(e.g. “Direct PC”), or other interfaces for coupling a computer systemto other computer systems. Client computer systems 35 and 37 are coupledto a LAN 33 through network interfaces 39 and 41, which can be Ethernetnetwork or other network interfaces. The LAN 33 is also coupled to agateway computer system 31 which can provide firewall and other Internetrelated services for the local area network. This gateway computersystem 31 is coupled to the ISP 7 to provide Internet connectivity tothe client computer systems 35 and 37. The gateway computer system 31can be a conventional server computer system. Also, the web serversystem 9 can be a conventional server computer system.

Alternatively, as well-known, a server computer system 43 can bedirectly coupled to the LAN 33 through a network interface 45 to providefiles 47 and other services to the clients 35, 37, without the need toconnect to the Internet through the gateway system 31.

FIG. 5B shows one example of a conventional computer system that can beused as a client computer system or a server computer system or as a webserver system. It will also be appreciated that such a computer systemcan be used to perform many of the functions of an Internet serviceprovider, such as ISP 5. The computer system 51 interfaces to externalsystems through the modem or network interface 53. It will beappreciated that the modem or network interface 53 can be considered tobe part of the computer system 51. This interface 53 can be an analogmodem, ISDN modem, cable modem, token ring interface, satellitetransmission interface (e.g. “Direct PC”), or other interfaces forcoupling a computer system to other computer systems. The computersystem 51 includes a processing unit 55, which can be a conventionalmicroprocessor such as an Intel Pentium microprocessor or Motorola PowerPC microprocessor. Memory 59 is coupled to the processor 55 by a bus 57.Memory 59 can be dynamic random access memory (DRAM) and can alsoinclude static RAM (SRAM). The bus 57 couples the processor 55 to thememory 59 and also to non-volatile storage 65 and to display controller61 and to the input/output (I/O) controller 67. The display controller61 controls a display on a display device 63, such as, for example, acathode ray tube (CRT) or liquid crystal display, in accordance with thepresent invention. The input/output devices 69 can include a keyboard,disk drives, printers, a scanner, and other input and output devices,including a mouse or other pointing device. The display controller 61and the I/O controller 67 can be implemented with conventional wellknown technology. A digital image input device 71 can be a digitalcamera which is coupled to an I/O controller 67 in order to allow imagesfrom the digital camera to be input into the computer system 51. Thenon-volatile storage 65 is often a magnetic hard disk, an optical disk,or another form of storage for large amounts of data. Some of this datais often written, by a direct memory access process, into memory 59during execution of software in the computer system 5 1. One of skill inthe art will immediately recognize that the terms “machine-readablemedium” and “computer-readable medium” includes any type of storagedevice that is accessible by the processor 55 and also encompasses acarrier wave that encodes a data signal.

It will be appreciated that the computer system 51 is one example ofmany possible computer systems which have different architectures. Forexample, personal computers based on an Intel microprocessor often havemultiple buses, one of which can be an input/output (I/O) bus for theperipherals and one that directly connects the processor 55 and thememory 59 (often referred to as a memory bus). The buses are connectedtogether through bridge components that perform any necessarytranslation due to differing bus protocols.

Network computers are another type of computer system that can be usedwith the present invention. Network computers do not usually include ahard disk or other mass storage, and the executable programs are loadedfrom a network connection into the memory 59 for execution by theprocessor 55. A Web TV system, which is known in the art, is alsoconsidered to be a computer system according to the present invention,but it may lack some of the features shown in FIG. 5B, such as certaininput or output devices. A typical computer system will usually includeat least a processor, memory, and a bus coupling the memory to theprocessor.

It will also be appreciated that the computer system 51 is controlled byoperating system software which includes a file management system, suchas a disk operating system, which is part of the operating systemsoftware. One example of an operating system software with itsassociated file management system software is the family of operatingsystems known as Mac® OS from Apple Computer, Inc. of Cupertino, Calif.,and their associated file management systems. The file management systemis typically stored in the non-volatile storage 65 and causes theprocessor 55 to execute the various acts required by the operatingsystem to input and output data and to store data in memory, includingstoring files on the non-volatile storage 65.

Virtualization of graphics resources has been described. Althoughspecific embodiments have been illustrated and described herein, it willbe appreciated by those of ordinary skill in the art that anyarrangement which is calculated to achieve the same purpose may besubstituted for the specific embodiments shown. This application isintended to cover any adaptations or variations of the presentinvention. The terminology used in this application with respect tographics is meant to include all environments that display images to auser. Therefore, it is manifestly intended that this invention belimited only by the following claims and equivalents thereof.

1. A computerized method of virtualizing graphics resources comprising:allocating, by a data processing system, a graphics resource to aplurality of graphics clients, wherein the graphics resource comprisesgraphics hardware; assigning an identifier to reference the graphicsresource in a command; sending the identifier to the graphics clientthat requested allocation of the graphics resource; associating theidentifier with an entry in a virtualization map containing informationabout the allocation of the graphics resource; processing commands foraccess to the graphics resource from the plurality of graphics clients,wherein processing commands comprises: locating the commands in acommand stream; replacing the identifier in the commands with an addressfor the graphics resource, wherein the identifier is different than theaddress; and resolving, by a graphics kernel, a conflict for thegraphics resource among the plurality of graphics clients.
 2. Thecomputerized method of claim 1, wherein locating the commands comprises:jumping to the commands using offset values specified in a series ofjump packets in the command stream.
 3. The computerized method of claim1, wherein resolving a conflict comprises: determining the graphicsresource has been used by a second graphics client when a first graphicsclient requests access to the graphics resource; and refreshing thegraphics resource with data for the first graphics client.
 4. Thecomputerized method of claim 3, wherein refreshing the graphics resourcecomprises: paging data in the graphics resource for the second graphicsclient into a backing store; and paging the data for the first graphicsclient from the backing store into the graphics resource.
 5. Thecomputerized method of claim 3, wherein refreshing the graphics resourcecomprises: paging the data for the first graphics client from a backingstore into a like graphics resource; and assigning the like graphicsresource an identifier that was previously assigned to the graphicsresource.
 6. The computerized method of claim 1, wherein processingcommands comprises: inserting a reference to a graphics hardwaresemaphore before a command that causes the conflict for the graphicsresource; and clearing the graphics hardware semaphore when the conflictis resolved.
 7. The computerized method of claim 1 further comprising:creating a command requesting access to the graphics resource using theidentifier.
 8. The computerized method of claim 7, wherein creating acommand comprises: inserting a jump packet to a next command containingan identifier.
 9. A non-transitory machine-readable storage mediumstoring executable instructions to cause a processing system to performa method comprising: allocating a graphics resource to a plurality ofgraphics clients, wherein the graphics resource comprises graphicshardware; assigning an identifier to reference the graphics resource ina command; sending the identifier to the graphics client that requestedallocation of the graphics resource; associating the identifier with anentry in a virtualization map containing information about theallocation of the graphics resource; processing commands for access tothe graphics resource from the plurality of graphics clients, whereinprocessing commands comprises: locating the commands in a commandstream; replacing the identifier in the commands with an address for thegraphics resource, wherein the identifier is different than the address;and resolving, by a graphics kernel, a conflict for the graphicsresource among the plurality of graphics clients.
 10. The non-transitorymachine-readable storage medium of claim 9, wherein locating thecommands comprises: jumping to the commands using offset valuesspecified in a series of jump packets in the command stream.
 11. Thenon-transitory machine-readable storage medium of claim 9, whereinresolving a conflict comprises: determining the graphics resource hasbeen used by a second graphics client when a first graphics clientrequests access to the graphics resource; and refreshing the graphicsresource with data for the first graphics client.
 12. The non-transitorymachine-readable storage medium of claim 11, wherein refreshing thegraphics resource comprises: paging data in the graphics resource forthe second graphics client into a backing store; and paging the data forthe first graphics client from the backing store into the graphicsresource.
 13. The non-transitory machine-readable storage medium ofclaim 11, wherein refreshing the graphics resource comprises: paging thedata for the first graphics client from a backing store into a likegraphics resource; and assigning the like graphics resource anidentifier that was previously assigned to the graphics resource. 14.The non-transitory machine-readable storage medium of claim 9, whereinprocessing commands comprises: inserting a reference to a graphicshardware semaphore before a command that causes the conflict for thegraphics resource; and clearing the graphics hardware semaphore when theconflict is resolved.
 15. The non-transitory machine-readable storagemedium of claim 9 further comprising: creating a command requestingaccess to the graphics resource using the identifier.
 16. Thenon-transitory machine-readable storage medium of claim 15, whereincreating a command comprises: inserting a jump packet to a next commandcontaining an identifier.
 17. A computer system comprising: a processorcoupled to a memory through a bus; a graphics processor coupled to theprocessor through the bus and associated with graphics resources; and akernel driver executed by the graphics processor to cause the graphicsprocessor to allocate a graphics resource to a plurality of graphicsclients, wherein the graphics resource comprises graphics hardware,assign an identifier to reference the graphics resource in a command,send the identifier to the graphics client that requested allocation ofthe graphics resource, associate the identifier with an entry in avirtualization map containing information about the allocation of thegraphics resource, process commands for access to the graphics resourcefrom the plurality of graphics clients, wherein the kernel driver, whenprocessing commands, further causes the graphics processor to: locatethe commands in a command stream, replace the identifier in the commandswith an address for the graphics resource, wherein the identifier isdifferent than the address, and resolve, by the kernel driver, aconflict for the graphics resource among the plurality of graphicsclients.
 18. The computer system of claim 17, wherein the kernel driver,when locating the commands, further causes the graphics processor tojump to the commands using offset values specified in a series of jumppackets in the command stream.
 19. The computer system of claim 17,wherein the kernel driver, when resolving a conflict, further causes thegraphics processor to determine the graphics resource has been used by asecond graphics client when a first graphics client requests access tothe graphics resource, and refresh the graphics resource with data forthe first graphics client.
 20. The computer system of claim 19, whereinthe kernel driver, when refreshing the graphics resource, further causesthe graphics processor to page data in the graphics resource for thesecond graphics client into a backing store, and page the data for thefirst graphics client from the backing store into the graphics resource.21. The computer system of claim 19, wherein the kernel driver, whenrefreshing the graphics resource, further causes the graphics processorto page the data for the first graphics client from a backing store intoa like graphics resource, and assign the like graphics resource anidentifier that was previously assigned to the graphics resource. 22.The computer system of claim 17, wherein the kernel driver, whenprocessing commands, further causes the graphics processor to insert areference to a graphics hardware semaphore before a command that causesthe conflict for the graphics resource, and clear the graphics hardwaresemaphore when the conflict is resolved.
 23. The computer system ofclaim 17 further comprising: a client driver executed by the processorfrom the memory to cause the processor to create a command requestingaccess to the graphics resource using the identifier.
 24. The computersystem of claim 23, wherein the client driver further causes theprocessor to insert a jump packet to a next command containing anidentifier when creating a command.
 25. A data processing systemcomprising: means for allocating, by a hardware device, a graphicsresource to a plurality of graphics clients, wherein the graphicsresource comprises graphics hardware; means for assigning an identifierto reference the graphics resource in a command; means for sending theidentifier to the graphics client that requested allocation of thegraphics resource; means for associating the identifier with an entry ina virtualization map containing information about the allocation of thegraphics resource; means for processing commands for access to thegraphics resource from the plurality of graphics clients, wherein themeans for processing commands comprises: means for locating the commandsin a command stream; means for replacing the identifier in the commandswith an address for the graphics resource, wherein the identifier isdifferent than the address; and means for resolving, by a graphicskernel, a conflict for the graphics resource among the plurality ofgraphics clients.